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Abstract 

A Runway Incursion Prevention System 
(RIPS) and additional incursion detection algorithm 
were adapted for general aviation operations and 
evaluated in a simulation study at the National 
Aeronautics and Space Administration (NASA) 
Langley Research Center (LaRC) in the fall of 
2005. RIPS has been designed to enhance surface 
situation awareness and provide cockpit alerts of 
potential runway conflicts in order to prevent 
runway incidents while also improving operational 
capability. The purpose of the study was to 
evaluate the airborne incursion detection algorithms 
and associated alerting and airport surface display 
concepts for general aviation operations. This 
paper gives an overview of the system, simulation 
study, and test results. 

Introduction 

Runway incursions are a serious aviation 
safety hazard, particularly for general aviation (GA) 
operations. According to the Federal Aviation 
Administration (FAA) [1], during the four year 
period from fiscal year (FY) 2001 through FY 
2004, there were approximately 257 million aircraft 
operations and 1,395 runway incursions reported at 
United States towered airports - approximately 5.4 
runway incursions for every one million operations. 
General aviation accounted for 74 percent of these 
incursions but only 57 percent of the operations. 
Seventy-six percent of the most severe incursions 
(1 14 of 150 incursions) involved at least one GA 
aircraft. Five incursions resulted in collisions, with 
four of these collisions involving GA aircraft. 

These statistics do not consider incidents that occur 
at non-towered airports. 

Current FAA initiatives are targeting 
reductions in the severity, number, and rate of 
runway incursions by implementing a combination 


of technology, infrastructure, procedural, and 
training interventions [1]. None of these initiatives 
involve technology solutions for the aircraft. 

The National Transportation Safety Board 
(NTSB) also considers runway incursions to be a 
serious aviation safety hazard, listing runway 
incursion prevention as one of their “most wanted” 
transportation safety improvements [2]. The NTSB 
specifically recommends that the FAA implement 
technology that “provides a direct warning 
capability to flight crews” [3]. 

NASA developed a Runway Incursion 
Prevention System (RIPS) for commercial aviation 
operations to improve airport safety by providing 
supplemental surface situation awareness 
information and guidance cues, and alerts of 
runway conflicts and route deviations directly to the 
flight crew. The system was evaluated in several 
flight tests and simulation studies [4] [5] [6]. 

RIPS was adapted for GA operations and 
integrated with NASA’s GA synthetic vision 
system. A piloted simulation study was conducted 
at NASA LaRC to evaluate RIPS for GA operations 
focusing on analysis of the incursion detection 
algorithms and display concepts. This paper will 
present an overview of the system, description of 
the simulation study, and reporting of test results. 

System Description 

Simulation Facility 

Flight Deck Simulator 

NASA LaRC’s Integration Flight Deck (IFD) 
transport category fixed-based high-fidelity flight 
simulator was used for this study because of its 
visual, tactile, and audio capabilities. The IFD was 
adapted to emulate a Cessna 206 (C-206) GA 
aircraft (herein referred to as the ownship). A basic 
six degree of freedom non-linear simulation model 
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of the C-206 and representative control force and 
braking models were used. In order to avoid 
distractions, extraneous displays and gauges were 
covered or turned off during data collection. This 
configuration was used successfully in a previous 
GA simulation [7]. 


As shown in Figure 1, an electronic research 
display (RD) was installed on the instrument panel 
directly in front of the left seat and control yoke. 
The RD was composed of two 10.4” liquid crystal 
displays and was capable of displaying two separate 
digital displays, side-by-side. An electronic flight 
bag (EFB) display, located to the left of the RD, 
was used to present the airport surface map display 
concepts described below. This display was 10.4” 
diagonal with a resolution of 1280 x 1024 pixels. 
The collimated out-the-window scene provided a 
200 degree horizontal by 40 degree vertical field- 
of-view at 26 pixels per degree resolution. 


Traffic position data was “broadcast” at a 1 Hz 
rate. No additional latency in traffic position was 
used. Ownship position data was updated at a 20 
Hz rate. Positional error was not introduced into 
these data. 



Figure 1. IFD Flight Deck 

Air Traffic Control Simulation 

An air traffic control (ATC) simulation was 
created to establish a realistic terminal area 
environment. Approach and tower ATC 
instructions and traffic requests and replies were 
prerecorded using different voices. The messages 
were then played through the flight deck speaker 
system when the ownship and simulated traffic 
reached specified locations. 


Research Displays 

This simulation study was geared for low end 
GA aircraft; therefore, standard round dial 
instrumentation was used in conjunction with 
various airport surface map formats and alerting. 
The round dial displays were shown on the RD 
located in front of the evaluation pilot (EP) (see 
Flight Deck Simulator section). When required, an 
airport surface map was displayed on the EFB 
located to the left of the round dial display. The 
surface map was generated using a Reno 
International (RNO) airport geographic database 
developed to RTCA standards [8]. The map scale 
was set to 2.5 nm for the airborne scenarios and 1.5 
nm for the ground based scenarios. Audible alerts 
sounded through the flight deck speaker system. 

Seven display conditions were evaluated 
during the course of the study as described below. 

Baseline (B) 

The Baseline (B) display condition consisted 
of a set of six GA instruments (airspeed, attitude, 
altitude, vertical speed, directional gyro, and turn 
coordinator) plus manifold pressure and instrument 
landing system data. All instruments were three 
inches in diameter and configured on the RD as 
shown in Figure 2. 



Figure 2. Baseline Display Condition 

Baseline with Surface Map and Ownship 
(BMO) 

The BMO display condition consisted of the 
baseline round dials displayed on the RD with the 
addition of a plan view surface map displayed on 
the EFB. This version of the surface map displayed 
an airport layout along with ownship position. 
Traffic was not shown. Incursion alerting was not 
part of this condition. 
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Baseline with Surface Map and Ownship 
and Traffic (BMOT) 

The BMOT display condition was equivalent 
to the BMO condition with the addition of traffic on 
the surface map (Figure 3). Traffic was indicated by 
dark blue chevrons when on the ground and cyan 
chevrons when airborne. Incursion alerting was 
not part of this condition. 



Figure 3. Plan View Surface Map 

Baseline with Audible Incursion Alerts (BA) 

The BA display condition consisted of the 
baseline round dials displayed on the RD and 
audible runway incursion alerts sounded in the 
flight deck when a potential conflict was detected. 

Baseline with Surface Map and Ownship 
and Audible Incursion Alerts (BAMO) 

The BAMO display condition was equivalent 
to the BMO condition with the addition of audible 
runway incursion alerts. 

Baseline with Surface Map and Ownship, 
Traffic, and Incursion Alerts (BAMOT) 

The BAMOT display condition was equivalent 
to the BMOT condition with the addition of both 
audible and graphical runway incursion alerts. The 
alert implementation is described in detail below in 
the Runway Incursion Alerting section. 


Baseline with Perspective Surface Map 
(BRIPS) 

The BRIPS display condition consisted of the 
baseline round dials displayed on the RD with a 
perspective surface map displayed on the EFB. The 
map showed a perspective track-up airport layout 
with ownship and traffic locations, ATC 
instructions (including the approved taxi route and 
hold short locations), and incursion alerts (Figure 
4). Audible incursion alerts were also sounded. 

Audible route deviation and crossing hold 
alerts were also generated. Route deviation alerts 
were generated if ownship left its assigned path 
during taxi. Crossing hold alerts were generated if 
ownship crossed a hold line without clearance. 



Figure 4. Perspective Surface Map 

Runway Incursion Alerting 

The runway was monitored for potential 
incursions any time the ownship was to enter the 
runway - during final approach and landing, take- 
off roll, and taxi crossing. If an incursion was 
predicted, audible and graphical alerts were 
generated and presented to the flight crew. These 
aircraft-generated alerts could also be data linked to 
ATC so the pilots and controllers have the same 
awareness. Currently, conflict resolution advisories 
are not provided. 
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Two different incursion detection algorithms 
were evaluated during the simulation study. 

The Runway Safety Monitor (RSM) incursion 
detection algorithm [9] uses a generic approach for 
detecting and generating incursion alerts and is not 
designed to detect only specific incursion scenarios. 
The RSM monitors traffic that enters a three- 
dimensional virtual protection zone around the 
runway that is being used by the ownship. 

Incursion detection is based on the operational state 
of the ownship and traffic, as well as other criteria 
(separation and closure rate), to avoid false alerts. 
Identification, position, and altitude data is used to 
track the traffic in the protection zone. Traffic data 
projections are calculated within RSM since, from 
flight test experience, reliable position updates are 
not received at consistent intervals. RSM generates 
a Warning alert, which occurs when a runway 
incursion is detected and evasive action is required 
to avoid a potential collision. Information provided 
with each alert includes identification of the 
incurring traffic and separation distance to potential 
conflict. RSM was developed for NASA by 
Lockheed Martin. 

The PathProx™ detection algorithm [10] is 
designed to handle over 40 specific runway 
incursion scenarios. Alerts are issued based on the 
states of the ownship and traffic and on conditions 
including position, speed, and track angle. 
PathProx™ generates two types of alerts analogous 
to the Traffic Alert and Collision Avoidance 
System (TCAS) approach. A Caution alert informs 
the flight crew of a potential incursion or an 
incursion where the conflict does not yet require 
evasive action. The crew can take evasive action, 
however, at their discretion. PathProx™ also 
generates Warning alerts when immediate evasive 
action is required. Information provided with each 
alert includes identification of the incurring traffic, 
the associated runway, and separation distance 
between the traffic and ownship. PathProx™ was 
developed by Rannoch Corporation. 

For this test, the incursion alerts were 
presented to the flight crew both visually on the 
surface map and audibly, if required by the current 
display condition. The alert phrases were designed 
to provide descriptive information regarding the 
location of the incurring traffic, e.g. “Warning, 
Traffic Departing 25”, “Caution, Traffic 34R”. The 


textual forms of these alerts were presented on the 
surface map. Also, the traffic symbol representing 
the incurring aircraft was enlarged, changed color 
(yellow for Caution and red for Warning), and was 
highlighted by a target designator box. The 
identification tag was also highlighted. In the event 
the incurring traffic symbol was not shown because 
of the display scale, a symbol was pegged on the 
edge of the display in the direction of the traffic on 
the perspective surface map only. The distance to 
conflict was shown beneath the ownship symbol. 
Figure 4 shows and example of a Warning alert. 

Test Method 

Data collection occurred for several different 
scenarios and test conditions as described below. 

Test Scenarios 

Five incursion scenarios were evaluated during 
the simulation. The EP was trained to abort if a 
warning alert was given during departure, go- 
around if a warning alert was given on approach, 
and stop if a warning alert was given during taxi. 
The EP was not required to take evasive action 
when a caution alert was issued. 

A traffic pattern was established to emulate a 
representative GA traffic flow at RNO. Alternating 
arrivals and departures were on Runway 34R with 
interleaving departures on Runway 25. Traffic 
traveled through the intersection of runways 25 and 
34R every minute. The incurring traffic was 
interleaved into this traffic flow. 

Every effort was made to produce similar 
timing for the scenarios; however, a certain amount 
of variability was naturally introduced due to the 
maneuvering conducted by the EP (i.e., approach 
speed, taxi speed, etc.). 

Scenario 1 - arrival/take-off hold 

Scenario 1 tested the incursion situation where 
an aircraft was in position and holding for departure 
clearance while another aircraft was approaching 
the same runway for a landing (Figure 5). The 
ownship was approaching Runway 34R, 3 nm from 
the threshold at 1010’ above field level (AFL) and 
at an indicated airspeed of 90 kts. The EP was 
instructed to land. The traffic was initially at the 
34R hold line nearest to the runway threshold. The 
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traffic was cleared to taxi into position on the active 
runway when the ownship was 2 nm from the 
threshold and held in that location awaiting 
departure clearance. 
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Figure 5. Scenario 1 Configuration 


Scenario 2 - departure/intersection 
departure 

Scenario 2 tested the case where one aircraft 
was departing as another aircraft entered the 
runway for an intersection departure. The ownship 
was initially on Taxiway C approximately 500’ 
from the Runway 34R threshold traveling at 8 kts. 
The traffic was located at the 34R hold line on 
Taxiway L for an intersection departure. (Taxiway 
L is approximately 3000’ from Runway 34R 
threshold.) The ownship was cleared for departure. 


Once the ownship began its departure (on runway 
heading and traveling greater than 10 kts), the 
traffic taxied across the hold line and entered 
Runway 34R without clearance. 

Scenario 3 - arrival/departure 

Scenario 3 was a crossing runway scenario 
with one aircraft landing at the same time another 
aircraft was departing on an intersecting runway. 
The ownship was approaching Runway 34R, 3nm 
from the threshold at 1010’ AFL and at an indicated 
airspeed of 90 kts. The EP was instructed to land 
and stop on the runway. The traffic was initially on 
Taxiway L near the Runway 25 hold line at the 
threshold. The traffic was cleared to taxi into 
position on Runway 25 while the ownship was on 
approach to Runway 34R. When the ownship 
crossed the threshold, the traffic began its departure 
on Runway 25 without receiving clearance. 

Scenario 4 - departure/departure 

Scenario 4 was another crossing runway 
scenario where two aircraft were departing 
simultaneously on crossing runways. The ownship 
was initially on Taxiway C approximately 500’ 
from the Runway 34R threshold traveling at 8 kts. 
The traffic was initially on Taxiway L near the 
Runway 25 hold line at the threshold. The traffic 
was cleared into position on Runway 25 while the 
ownship taxied on C. The EP was cleared for 
departure on Runway 34R. Once the ownship 
began its departure (on runway heading and 
traveling greater than 10 kts), the traffic began its 
departure without receiving clearance. 

Scenario 5 - taxi crossing/departure 

Scenario 5 was designed to evaluate the taxi 
crossing situation. The ownship began on the ramp 
at the Mercury Air Center facing Taxi way L. The 
traffic began on Taxiway L at the Runway 25 hold 
line at the runway threshold. The traffic was 
cleared to taxi into position while the ownship 
taxied out of the ramp. The ownship was cleared to 
taxi to Runway 34R, with instructions to cross 
Runway 25 (without stopping). When the ownship 
crossed the centerline of Taxiway L, the traffic 
began its departure without receiving clearance. 

Evaluation Pilots 

Sixteen GA pilots served as participants in the 
experiment with an equal distribution of flying 
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experience used to represent a cross-section of the 
Part 91 pilot population: low-time (<400 hours) 
visual flight rules (VFR), high-time (>400 hours) 
VFR, low-time (<1000 hours) instrument-rated, and 
high-time (>2000 hours) instrument-rated. 

Test Matrix 

The testing was conducted in two phases. A 
“rare event” study evaluated pilot reaction to a 
runway incursion event with a given display 
condition and incursion detection algorithm without 
expectation on the part of the subject as to the true 
intent of the study. Four display conditions (BMO, 
BMOT, BAMO, BAMOT) were evaluated across 
subjects (each EP was given only one display 
condition). Eighteen runs were randomly flown 
that consisted of six different approaches in varying 
day time visual meteorological conditions (VMC) 
and instrument meteorological conditions (IMC) 
visibility conditions. The intent of these runs was 
to provide sufficient variety and task demands to 
hide the true focus of the study. The final run in the 
block was the runway incursion event (Scenario 1 
above) conducted in VMC conditions (3 nm 
visibility and 1000’ ceiling). Scenario 1 - the 
arrival/take-off hold incursion - was selected 
because of its prevalence in runway incursion 
incident and accident statistics. A wind profile was 
introduced to add workload to the tasks. 

Following the rare event study phase, a 
usability study evaluated the display concepts 
effectiveness for runway incursion prevention. 

Each EP evaluated all five incursion scenarios with 
the seven display conditions. The first run for each 
scenario group used the baseline display condition. 
All of the runs that provided alerting used the RSM 
incursion detection algorithm as the alert source. In 
this study phase, the subjects had an expectation for 
the study intent (i.e., runway incursions) but they 
did not know the scenario before the initial run. 
Another purpose of the usability study was to 
evaluate the RSM and PathProx™ incursion 
detection algorithms for GA operations. Each EP 
evaluated both algorithms using all five incursion 
scenarios and the BRIPS display condition. All 
runs in the usability study were conducted in VMC 
conditions (3 nm visibility and 1000’ ceiling) 
without winds. 


Procedure 

Each EP participated in an extensive briefing 
and training session prior to data collection that was 
principally designed to mask the focus of the 
experiment (runway incursion prevention) for the 
rare event testing. The EP was trained on the 
incursion alerting system prior to the rare event 
testing only if the display condition evaluated 
included alerting; otherwise, the training was 
conducted before the usability study. Before each 
run, the pilots were briefed on the run conditions, 
e.g. approach or departure, visibility, alerting 
system selected (for usability study), and displays 
available. Post-run, post-block, and post-test 
questionnaires were administered when required. 
Audio, video, and digital data were also recorded. 

Results 

A summary of quantitative and qualitative 
results is presented for the rare event testing and 
usability study. All data is referenced from the 
center of gravity of the aircraft. 

Rare Event Testing 

As described above, each EP received 1 8 
various approach tasks before being presented with 
the runway incursion event (Scenario 1). 

The FAA runway incursion severity rating [1], 
described below, was used to categorize the runway 
incursion incident data from this study. 

• Category A - Separation decreases, extreme 
action taken to narrowly avoid collision, or 
collision occurs; 

• Category B - Separation decreases, 
significant potential for collision; 

• Category C - Separation decreases, ample 
time and distance to avoid collision; 

• Category D - Little or no chance of collision 
but meets definition of runway incursion. 

Fourteen scenarios resulted in the less 
hazardous Category C and D incursions, one 
resulted in a Category A incursion, and one resulted 
in a Category B incursion. 

The 14 less hazardous Category C and D 
incursions were mitigated by the EPs by conducting 
a go-around and gaining separation from the traffic. 
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The Category A runway incursion occurred 
with the EP flying the BMOT display concept. 
Despite the traffic indications on the surface map 
and out-the-window visuals, the EP demonstrated 
no awareness of the runway traffic and over-flew 
the traffic and landed. 

The Category B incident occurred when the EP 
over-flew the runway traffic (at 146’ AFL) before 
conducting a go-around. The EP was aware of the 
incursion after having received an audible alert 
(BAMO display concept) but continued to descend 
to visually acquire the traffic to confirm the alert. 
This incident would have been classified as a 
Category D incursion if the EP had initiated the go- 
around at first awareness of the alert. 

No statistically significant differences were 
found between the display concepts for the distance 
to the incurring traffic when a go-around was 
initiated by the pilots or for the EP’s reaction time 
from the incursion event occurrence (based on a 
Multivariate Analysis of Variance (MANOVA) test 
p > 0.05). 

Data on both incursion detection algorithms 
were collected during the incursion events; 
however, only the alerts generated by the RSM 
algorithm were displayed to the EP, when the 
display condition included alerting. For all 
incursion runs, the RSM issued an alert when the 
ownship was an average distance of 4060’ and 
approximately 24 seconds from the traffic. 
PathProx™ computed that a Caution alert was 
necessary when ownship was 5836’ and 
approximately 35 seconds from the traffic and a 
Warning alert was necessary when ownship was 
4234’ and approximately 25 seconds from the 
traffic. (Data from two EPs were omitted from 
these averages because the pilots were initially 
heading to the wrong runway and turned toward 
Runway 34R within 1 . 1 nm. Even though alerts 
were generated the results were skewed.) A 
MANOVA showed a significant effect between the 
incursion detection algorithms tested (F(4,42) = 
190.163, p< .0001). A subsequent ANOVA on the 
dependent variables revealed that the PathProx™ 
Caution alert was generated significantly earlier 
(F(2,42) = 6.839) and at a greater distance from the 
incursion aircraft (F(2,42) = 7.302) than from either 
the PathProx™ Warning alert or the RSM Warning 
alert. There were no significant differences 


between the PathProx™ Warning alert and RSM 
Warning alert. 

Of the 16 runs, eight EPs initially became 
aware of the incursion traffic before the alert would 
have occurred by viewing out the window; five 
visually acquired the traffic out the window after an 
incursion alert, if used, would have occurred; one 
saw the traffic on the surface map well before the 
alert occurred; and two EPs did not see the traffic at 
all (the category A and B incursions described 
above). As shown in Figure 6, the incursion traffic 
was acquired sooner when the EP was provided 
with a traffic display on the surface map and/or 
incursion alerts, but the differences were not 
statistically significant at the a = 0.05 level. The 
data for the two EPs that did not acquire the traffic 
and the two EPs that headed to the incorrect runway 
were omitted from Figure 6. 
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Figure 6. Initial Traffic Awareness 


For those displays that had alerting (BAMO, 
BAMOT), there were no significant differences in 
timeliness of the alerting in terms of being able to 
take evasive action. However, when pilots were 
asked to rate all four display concepts on the 
perceived efficacy of the alerts (F(3,16) = 10.948) 
and the additional safety value added (F(3,16) = 
8.814) an ANOVA revealed a significant effect 
between the displays (p < 0.01). Subsequent post- 
hoc Student Newman Kuels tests showed that pilots 
reported that the BMO display condition was 
significantly poorer than the other three display 
conditions which were not significantly different 
from each other. 
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Usability Study 

All test runs conducted during the usability 
study included incursion events. During each test 
run, data were simultaneously collected on the 
performance of both incursion detection algorithms; 
however, only one method was chosen for display. 

The EPs evaluated all five incursion scenarios 
with the seven display conditions to determine the 
effectiveness of the display for runway incursion 
prevention. The RSM was used as the alert source 
for this testing phase. Although the EPs were aware 
that each run would contain an incursion event, they 
were not told the type of incursion before the initial 
run for each scenario. The initial run evaluated the 
baseline display condition. This was done to 
determine if the EP would visually acquire the 
incursion traffic out the window since the surface 
map and alerting were not available. 

The EPs also evaluated the RSM and 
PathProx™ incursion detection algorithms for GA 
operations using the five incursion scenarios and 
the BRIPS display condition. 

Quantitative Results 

A total of 612 test runs were completed. Since 
the BMO and BAMO display conditions provided 
similar alerting information to the B and BA display 
conditions, a limited number of BMO and BAMO 
test runs were conducted in the interest of time. 

Data was not analyzed for 16 runs due to missing 
data files, yielding a total of 596 runs. Alerts were 
required for display on 432 test runs. During these 
432 runs, the RSM was chosen as the alert source 
80 percent of the time (347 runs). 

RSM generated alerts on 533 runs. RSM did 
not alert on 60 runs due to the maneuver taken by 
the EP. For example, the EP may have acquired the 
traffic out the window and conducted a go-around 
before the alerting criteria were met. RSM did not 
alert on three runs due to the scenario timing (no 
incursion event). For the PathProx™ algorithm, 
Caution alerts were only possible on 243 runs and 
of these, alerts were generated on 157 runs. Of the 
possible 596 test runs, PathProx™ generated 
warning alerts on 352 runs. The 244 runs in which 
PathProx™ did not alert were generally due to the 
maneuvering performed by the EP. For instance, 
when RSM alerting was provided, the pilot 
executed an aircraft maneuver and this generally 


occurred before the PathProx™ alerts were 
generated. Detailed PathProx™ analysis was not 
possible because specific alerting criteria is 
proprietary. 

Scenario 1 arrival/take-off hold results - For 
the Baseline display (B) condition, all EPs acquired 
the incursion traffic out the window for Scenario 1 
when approximately 1 nm from the runway 
threshold. As a result, the average go-around 
initiation point was 443’ AFL and 5240’ 
(approximately 28 seconds) from the traffic. The 
EP initiated a go-around after the PathProx™ 
Caution alert would have occurred on average (475’ 
AFL and 5641’ - approximately 34 seconds - from 
the traffic) but before either the RSM (409’ AFL 
and 4272’ - approximately 24 seconds - from the 
traffic) or PathProx™ (392’ AFL and 41 17’ - 
approximately 25 seconds - from the traffic) 
Warning alert would have occurred. 

Post-run data analysis on all Scenario 1 test 
runs during the usability study showed that the 
PathProx™ Caution alert was generated when the 
ownship was at approximately 608’ AFL and 5756’ 
(approximately 36 seconds) from the incurring 
traffic. Both the RSM (488’ AFL and 4121’ — 
approximately 25 seconds - to traffic) and 
PathProx™ (492’ AFL and 4173’ - approximately 
25 seconds - to traffic) Warning alerts were 
generated at essentially the same moment. It should 
be noted that during the test runs in which alerts 
were provided, the EP was asked to continue the 
maneuver until the alert was received for evaluation 
purposes. 

Scenario 2 departure/intersection departure 
results - Six EPs (37.5 percent) did not acquire the 
incursion traffic visually for Scenario 2 or saw the 
traffic too late to abort the departure and actually 
over-flew the runway traffic when using the 
Baseline (B) display condition. 

Figure 7 shows when the EP began to abort the 
departure to avoid a conflict with the incursion 
traffic. A greater safety margin resulted when alerts 
were provided (BA, BAMO, BAMOT, and BRIPS). 
The EP also aborted the departure for all of these 
runs. The departure was aborted later when alerting 
was not provided (B, BMO, and BMOT), although 
still with enough time to stop prior to reaching the 
traffic. Also, as discussed above, the EP actually 



took off on six runs and over- flew the traffic, using 
the Baseline display. 


For this departure scenario, the RSM Warning 
alert (ownship at 28 kts ground speed and 2638’ - 
approximately 21 seconds - from Taxi way L) was 
generated before and at a lower ground speed than 
the PathProx™ Warning alert (41 kts ground speed 
and 2452’ - approximately 19 seconds - from 
Taxi way L); however, both provided ample time to 
abort and stop before reaching the incurring traffic. 
PathProx™ Caution alerts are not generated on 
departure. 
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Figure 7. Scenario 2 Abort Initiation 


Scenario 3 arrival/departure results - For the 
Baseline (B) condition, 13 EPs (81 percent) did not 
acquire the incursion traffic visually for Scenario 3. 
Due to the scenario design; however, 14 EPs landed 
and stopped before reaching the crossing runway. 
One EP landed but taxied through the intersection 
as the traffic was departing from crossing Runway 
25. One EP conducted a go-around. 

On average, the RSM Warning alert (1303’ - 
approximately 1 1 seconds - to Runway 25) was 
generated slightly before the PathProx™ Warning 
alert (1181’ - approximately 9 seconds - to Runway 
25). Both algorithms alerted at or slightly before 
touchdown. PathProx™ Caution alerts were not 
generated for this scenario. 

For all Scenario 3 runs (124 total), a go-around 
was conducted just before touchdown ( 7’ AFL) on 
only 1 1 percent (14) of the runs. 


Scenario 4 departure/departure results - Five 
EPs (31.3 percent) saw the incursion traffic visually 
and aborted the departure when using the Baseline 
(B) display condition. Nine EPs (62.5 percent) did 
not see the incursion traffic visually or saw the 
traffic too late to abort the departure. For these 
runs, the ownship came within an average distance 
of 458’ (range from 179’ to 795’) from the 
incursion traffic. The timing was early for two of 
the runs resulting in no incursion event; therefore, 
these data are not included in the analysis. 

Generally, a greater safety margin resulted 
when alerts were provided (BA, BAMO, BAMOT, 
and BRIPS) as shown in Figure 8. For runs in 
which the EP acquired traffic, the departure was 
aborted later when alerting was not provided (B and 
BMOT); however, with still enough time to stop 
before the crossing runway. 
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Figure 8. Scenario 4 Abort Initiation 


The RSM Warning alerts were generated 
before the PathProx™ Warning alerts for this 
departure scenario. When RSM alerts were 
presented, the EP typically reacted based on those 
alerts; therefore, PathProx™ alerts were sometimes 
not generated or generated after the abort maneuver. 
For an accurate assessment of PathProx™ 
performance, only the results from the runs in 
which the PathProx™ alerts were shown to the EP 
will be presented. For these runs, the PathProx™ 
Warning alerts were generated when the ownship 
was going 48 kts ground speed and was 1891’ and 
approximately 15 seconds from Runway 25. The 
RSM Warning alerts were generated when the 
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ownship was going 29 kts ground speed and was 
2216’ and approximately 18 seconds from Runway 
25. Again, PathProx™ Caution alerts are not 
generated on departure. 

Scenario 5 taxi crossing/departure results - 
Fourteen EPs acquired the incursion traffic out the 
window when using the Baseline (B) condition and 
stopped before reaching Runway 25. One EP did 
not see the traffic and actually crossed Runway 25 
in front of the departing traffic. One EP saw the 
departing traffic out the window but chose to cross 
Runway 25 anyway. 

The RSM algorithm is designed to provide 
early warning of an impending incursion during 
taxi, based on the aircraft taxi speed. If the ownship 
is traveling 8 kts or greater and is not slowing 
down, the alert will be generated before the aircraft 
reaches the hold line, providing sufficient distance 
to stop before crossing the hold line. As the taxi 
speed increases, the alert is generated when the 
ownship is a farther distance from the hold line. 

The alert is not generated until after the ownship 
crosses the hold line when the ownship is traveling 
less than 8 kts. The 8 kt threshold was used to 
prevent false or nuisance alerts as the ownship taxis 
toward a hold line. (As stated earlier, details on the 
PathProx™ implementation are proprietary.) 

As with scenario 4, the RSM alerts were 
generated before the PathProx™ alerts for this 
scenario. Therefore, for an accurate assessment of 
PathProx™, only the results from runs in which the 
PathProx™ alerts were given to the EP will be 
presented. For these runs, the RSM Warning alerts 
were generated when the ownship was an average 
distance of 27’ before the hold line and 197’ from 
the edge of Runway 25. Table 1 shows that the 
RSM alert was generated before reaching the hold 
line (negative value) when the taxi speed was 
greater than 8 kts. A positive value indicates the 
aircraft crossed over the hold line. The RSM 
alerting is proactive in preventing an incursion 
(crossing the hold line) in this situation. 

PathProx™ Caution alerts were generated when the 
ownship was an average of 18’ past the hold line 
and 152’ from the runway edge. PathProx™ 
Warning alerts were generated when the ownship 
was 41’ past the hold line and 129’ from the runway 
edge. 


Table 1. Scenario 5 RSM Alert Generation 


EP 

Ground 
Speed (kts) 

Distance to 
hold line (feet) 

1 

10.7 

-86 

1 

9.7 

-59 

2 

9.7 

-48 

3 

9.2 

-59 

4 

7.9 

12 

5 

7 

6 

6 

8.4 

-56 

7 

6 

7 

8 

13.4 

-17 

9 

10.3 

-70 

10 

8.4 

-48 

11 

8.3 

-48 

12 

7.5 

17 

13 

7.4 

14 

15 

6.5 

7 

16 

10.6 

-8 


Qualitative 

Post-run, post-block, and post-test 
questionnaires were administered when required. 
Ratings for most of the questions were given on a 
scale of 1 (low) to 10 (high). 

When asked to rate the effectiveness of the 
perspective surface map compared to the plan view 
map for prevention of runway incursions, the 
perspective map was rated slightly more effective 
(2.9). However, the location of the surface map 
was suboptimal and should be positioned closer to 
the pilots head-up field-of-view. The traffic 
presentation on the surface map was considered 
easily discernable (8.4). Most EPs (14 of 16) 
considered traffic presentation necessary to prevent 
runway incursions. The addition of traffic would 
provide increased (8.6) situation awareness (SA) 
over a surface map with only ownship location. 
Over half of the EPs (10 of 16) indicated graphical 
presentation of alerts on the surface map was 
necessary to prevent runway incursions. The 
addition of graphical alerts would provide increased 
(6.3) SA. The EPs considered the terms used for 
the incursion alerts (e.g. “Warning, Traffic 
Departing 25”) to be very effective (8.9 average 
rating for all phrases). 

For display conditions with alerting available, 
the EPs indicated that the incursion event would 
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most likely be brought to their attention first 
through audible alerting, then on the surface map, 
and lastly out the window. Nine of 16 EPs 
indicated that an audible alert alone would provide 
a minimally effective incursion prevention display, 
while five of 16 indicated a surface map with 
ownship and traffic would be minimally effective. 
However, all 16 EPs indicated a surface map with 
ownship and traffic in conjunction with an audible 
alert would be an optimal incursion prevention 
display. Table 2 shows, according to averaged EP 
ratings, that for all alerting display conditions, the 
caution and warning alerting system provided the 
greatest amount of runway incursion awareness. 

An ANOVA confirmed this conclusion with 
significant main effects (p < 0.0001) for display on 
pilot ratings of situation awareness of where they 
were located (F (6,90) = 143.956) and where other 
traffic and hazards were located (F(6,90) = 94.899). 


Table 2. Alerting Preference for Incursion 
Awareness and Resulting SA Improvement 


Alerting 

Type 

Warning only 

Caution & Warning 

# of EP 

SA 

# of EP 

SA 

BA 

6 

6.5 

10 

4.2 

BAMO 

6 

6.7 

10 

4.3 

BAMOT 

7 

7.7 

9 

5 

BRIPS 

7 

7.6 

9 

5.2 


All EPs indicated that both the RSM and 
PathProx™ alerting provided sufficient time to 
avoid a potential incursion conflict. Only six of 16 
EPs thought providing both caution and warning 
alerts was more effective in preventing runway 
incidents than a single warning alert. However, 
nine of 16 EPs indicated that the caution and 
warning system provided greater (4.4) situation 
awareness and provided more reaction time (even 
though this was not always the case). Eleven of 16 
EPs liked the idea of having a caution alert in 
conjunction with a warning to provide more 
evaluation and reaction time, i.e. a greater comfort 
level. For the scenarios evaluated, the EPs 
generally felt that providing caution and warning 
alerts on approach was most effective, while a 
warning alert alone was sufficient when on the 


airport surface (during departure and taxi). This 
was particularly true for Scenario 5. Many EPs 
indicated that the short caution alert had no benefit. 

The majority of EPs would like to be provided 
with maneuver guidance for conflict resolution in 
addition to runway incursion alerting on final 
approach (12 of 16) and when taxiing across a 
runway (nine of 16). Half of the EPs would like 
maneuver guidance on departure. 

In general, the EPs felt safer during runway 
incursion incidents when alerting was provided 
(B=2.1, BMO=2.75, BMOT= 5.4, BA=7.2, 
BAMO=7.4, BAMOT=9.3, and BRIPS=9.2). 
Analysis of pilot responses to their perceived safety 
and runway incursion prevention effectiveness 
support this conclusion with significant main effects 
found (i.e. for perceived safety (F(6,90) = 857.390) 
and runway incursion prevention effectiveness (F(6, 
90) = 188.793)). The addition of traffic was 
marginally beneficial when presented on a moving 
map display and was only effective when alerting 
was provided. A possible cause may be that pilots 
had to transition to out-the-window and were not 
focused on the head-down display. With alerting, 
the pilot is provided a cue to direct focus and 
attention to the head-down display to locate the 
incurring traffic. In fact, pilots rated having audible 
alerts (BA) and having alerts with a map with 
ownship but no traffic (BAMO) similarly for 
runway incursion prevention on almost all 
dependent variables measured. For the 
experimental scenarios tested, the moving map 
display revealed its utility only when traffic AND 
alerting were options. 

Summary 

A Runway Incursion Prevention System 
(RIPS) adapted for general aviation operations was 
evaluated in a simulation study in the fall of 2005. 
The purpose of the study was to evaluate the 
airborne incursion detection algorithms and 
associated alerting and airport surface display 
concepts for general aviation operations using a rare 
event study followed by a usability study. 

The results indicate that, during the rare event 
testing, most pilots were able to acquire the 
incurring traffic looking out the cockpit windows 
(in VMC conditions), even before incursion alerting 
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was activated. In the usability study, varying out- 
the-window detection of incurring traffic was noted 
and the importance of incursion detection 
algorithms was shown. 

The surface map with ownship (without other 
traffic) was rated as being significantly inferior to a 
surface map with traffic and/or incursion alerting 
for perceived safety value added. However, one 
pilot experienced a severe runway incursion risk for 
collision despite having traffic displayed on a 
surface map. The addition of traffic was marginally 
beneficial when presented on a surface map display 
and was only effective when alerting was provided. 
A possible cause may be that pilots had transitioned 
to out-the-window and were not focused on the 
head-down display to locate the incurring traffic. 
Pilots reported that the utility of the surface map 
would be significantly more effective if located 
higher on the instrument panel closer to the pilot’s 
head-up field-of-view. 

The results generally matches past research on 
commercial operations that the incursion alerts 
provided sufficient time to avoid a potential 
incursion conflict. Departures were generally 
aborted sooner when alerts were provided, resulting 
in greater safety margins. A surface map with 
ownship and traffic along with audible alerts was 
considered an optimal incursion prevention display, 
while an audible alert alone was considered a 
minimally effective display. Over half of the pilots 
would like maneuver guidance for conflict 
resolution in conjunction with incursion alerting. In 
general, the pilots felt substantially safer during 
runway incursion incidents with alerting onboard. 
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